Shapes from Echoes: Uniqueness from Point-to-Plane Distance Matrices

We study the problem of localizing a configuration of points and planes from the collection of point-to-plane distances. This problem models simultaneous localization and mapping from acoustic echoes as well as the notable "structure from sound" approach to microphone localization with unknown sources. In our earlier work we proposed computational methods for localization from point-to-plane distances and noted that such localization suffers from various ambiguities beyond the usual rigid body motions; in this paper we provide a complete characterization of uniqueness. We enumerate equivalence classes of configurations which lead to the same distance measurements as a function of the number of planes and points, and algebraically characterize the related transformations in both 2D and 3D. Here we only discuss uniqueness; computational tools and heuristics for practical localization from point-to-plane distances using sound will be addressed in a companion paper.Comment: 13 pages, 13 figure

Look, no Beacons! Optimal All-in-One EchoSLAM

We study the problem of simultaneously reconstructing a polygonal room and a trajectory of a device equipped with a (nearly) collocated omnidirectional source and receiver. The device measures arrival times of echoes of pulses emitted by the source and picked up by the receiver. No prior knowledge about the device's trajectory is required. Most existing approaches addressing this problem assume multiple sources or receivers, or they assume that some of these are static, serving as beacons. Unlike earlier approaches, we take into account the measurement noise and various constraints on the geometry by formulating the solution as a minimizer of a cost function similar to \emph{stress} in multidimensional scaling. We study uniqueness of the reconstruction from first-order echoes, and we show that in addition to the usual invariance to rigid motions, new ambiguities arise for important classes of rooms and trajectories. We support our theoretical developments with a number of numerical experiments.Comment: 5 pages, 6 figures, submitted to Asilomar Conference on Signals, Systems, and Computers Websit

Sound My Vision: Real-time Video Analysis On Mobile Platforms For Controlling Multimedia Performances

This paper presents Sound My Vision, an Android application for controlling music expression and multimedia projects. Unlike other similar applications which collect data only from sensors and input devices, Sound My Vision also analyses input video in real time and extracts low-level video features. Such a versatile controller can be used in various scenarios from entertainment and experimentation to live music performances, installations and multimedia projects. The application can replace complex setups that are usually required for capturing and analyzing a video signal in live performances. Additionally, mobility of smartphones allows perspective changes in sense that the performer can become either an object or a subject involved in controlling the expression. The most important contributions of this paper are selection of general and low-level video feature and the technical solution for seamless real-time video extraction on the Android platform

EchoSLAM: Simultaneous Localization and Mapping with Acoustic Echoes

We address the problem of jointly localizing a robot in an unknown room and estimating the room geometry from echoes. Unlike earlier work using echoes, we assume a completely autonomous setup with (near) collocated microphone and the acoustic source. We first introduce a simple, easy to analyze estimator, and prove that the sequence of room and trajectory estimates converges to the true values. Next, we approach the problem from a Bayesian point of view, and propose a more general solution which does not require any assumptions on motion and measurement model of the robot. In addition to theoretical analysis, we validate both estimators numerically

Structure from sound with incomplete data

In this paper, we consider the problem of jointly localizing a microphone array and identifying the direction of arrival of acoustic events. Under the assumption that the sources are in the far field, this problem can be formulated as a constrained low-rank matrix factorization with an unknown column offset. Our focus is on handling missing entries, particularly when the measurement matrix does not contain a single complete column. This case has not received attention in the literature and is not handled by existing algorithms, however it is prevalent in practice. We propose an iterative algorithm that works with pairwise differences between the measurements eliminating the dependence on the unknown offset. We demonstrate state-of-the-art performance both in terms of accuracy and versatility

Super Resolution Phase Retrieval for Sparse Signals

In a variety of fields, in particular those involving imaging and optics, we often measure signals whose phase is missing or has been irremediably distorted. Phase retrieval attempts to recover the phase information of a signal from the magnitude of its Fourier transform to enable the reconstruction of the original signal. Solving the phase retrieval problem is equivalent to recovering a signal from its auto-correlation function. In this paper, we assume the original signal to be sparse; this is a natural assumption in many applications, such as X-ray crystallography, speckle imaging and blind channel estimation. We propose an algorithm that resolves the phase retrieval problem in three stages: i) we leverage the finite rate of innovation sampling theory to super-resolve the auto-correlation function from a limited number of samples, ii) we design a greedy algorithm that identifies the locations of a sparse solution given the super-resolved auto-correlation function, iii) we recover the amplitudes of the atoms given their locations and the measured auto-correlation function. Unlike traditional approaches that recover a discrete approximation of the underlying signal, our algorithm estimates the signal on a continuous domain, which makes it the first of its kind. Along with the algorithm, we derive its performance bound with a theoretical analysis and propose a set of enhancements to improve its computational complexity and noise resilience. Finally, we demonstrate the benefits of the proposed method via a comparison against Charge Flipping, a notable algorithm in crystallography

A method for real-time detection of human fall from video

In this paper we present a method for real-time detection of human fall from video for support of elderly people living alone in their homes. The detection algorithm has four steps: background estimation, extraction of moving objects, motion feature extraction, and fall detection. The detection is based on features that quantify dynamics of human motion and body orientation. The algorithms are implemented in C++ using the OpenCV library. The method is tested using a single camera and 20 test video recordings showing typical fall scenarios and regular household behaviour. The experimental results show 90% of human fall detection accuracy

Combining Range and Direction for Improved Localization

Self-localization of nodes in a sensor network is typically achieved using either range or direction measurements; in this paper, we show that a constructive combination of both improves the estimation. We propose two localization algorithms that make use of the differences between the sensors’ coordinates, or edge vectors; these can be calculated from measured distances and angles. Our first method improves the existing edge-multidimensional scaling algorithm (E-MDS) by introducing additional constraints that enforce geometric consistency between the edge vectors. On the other hand, our second method decomposes the edge vectors onto 1-dimensional spaces and introduces the concept of coordinate difference matrices (CDMs) to independently regularize each projection. This solution is optimal when Gaussian noise is added to the edge vectors. We demonstrate in numerical simulations that both algorithms outperform state-of-the-art solutions

Point, Plane, Set, Match! Winning the Echo Grand SLAM

Some of the most important and challenging problems in science are inverse problems. They allow us to understand phenomena that cannot be measured directly. Inverse problems might not always have a unique or stable solution, or might not have any solution at all; in these cases, they are called ill-posed. An example of an inverse problem in room acoustics is simultaneous localization and mapping (SLAM) from sound. It is the central theme of this thesis, and we address it both from a theoretical and practical viewpoint. From a theoretical perspective, we show that our SLAM setup is ill-posed since the uniqueness condition is not satisfied. From a practical perspective, we propose methods to constrain and identify a solution set, and we use real experiments to confirm that such a constrained problem is stable. The acoustic SLAM problem consists of jointly reconstructing the geometry of a room and self-localizing. We show that it can be reformulated as the reconstruction of a set of points and planes from their pairwise distances; to solve the problem, we introduce point-to-plane distance matrices (PPDMs). Our motivation for PPDMs comes from the need of a robust localization system in indoor environments. Inspired by echolocation in animals, we approach the problem with the following analogy: Imagine that a bat loses the directivity in its sensing and becomes what we call an omnidirectional bat. It explores the room by moving randomly and listening to the echoes of its chirps. Can it still map the room and localize itself without any prior knowledge of the room geometry and its own trajectory? Our research shows that the answer is yes, but not in all rooms. We emulate such a bat using a device equipped with a collocated speaker and microphone. At different locations, the speaker emits a pulse and the microphone registers the room impulse response. The propagation times of the first-order echoes directly reveal the distances between the device and the walls. The problem is then to recover the locations of the device and walls from their pairwise distances; in the PPDM framework, its solution corresponds to the factorization of a PPDM. Though the most popular, SLAM is not the only application of PPDMs. In fact, our abstraction can be adapted to solve other problems in computer vision and signal processing which appear quite different at first glance, but they all rely on the factorization of a low-rank matrix. One famous example of such a problem is structure from motion (SFM), and aims at recovering a scene geometry and camera motions from images. A similar example in acoustics is called structure from sound (SFS), and concerns the joint localization of sensors and acoustic events. In both SFM and SFS, the idea is to factor a low-rank measurement matrix into the product of a projection and coordinate matrix. Finally, we can envision problems in which the projection matrix from SLAM, SFM or SFS is known, and the goal is to recover the coordinate matrix only. To solve these problems, we introduce another low-rank matrix named coordinate difference matrix (CDM). Possible applications include phase retrieval, where the core problem can be stated as the recovery of points from their unlabeled and noisy pairwise differences, and optimal tournament design, where we rely on CDMs to devise an active learning algorithm from pairwise comparisons. In a nutshell, this thesis revolves around the theory, algorithms and applications of PPDMs and CDMs

Coordinate Difference Matrices

In many problems such as phase retrieval, molecular biology, source localization, and sensor array calibration, one can measure vector differences between pairs of points and attempt to recover the position of these points; this class of problems is called vector geometry problems (VGPs). A closely related field studies distance geometry problems (DGPs), where only the Euclidean distance between pairs of points is available. This has been extensively studied in the literature and is often associated with Euclidean distance matrices (EDMs). Although similar to DGPs, VGPs have received little attention in the literature; our goal is to fill in this gap and introduce a framework to solve VGPs. Inspired by EDM-related approaches, we arrange the differences in what we call a coordinate difference matrix (CDM) and introduce a methodology to reconstruct a set of points from CDM entries. We first propose a reconstruction scheme in 1D and then generalize it to higher dimensions. We show that our algorithm is optimal in the least-squares sense, even when we have only access to partial measurements. In addition, we provide necessary and sufficient conditions on the number and structure of measurements needed for a successful reconstruction, as well as a comparison with EDMs. In particular we show that compared to EDMs, CDMs are simpler objects, both from an algorithmic and a theoretical point of view. Therefore, CDMs should be favored over EDMs whenever vector differences are available. In the presence of noise, we provide a statistical analysis of the reconstruction error. Finally, we apply the established knowledge to five practical problems to demonstrate the versatility of this theory and showcase the wide range of applications covered by the CDM framework